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I i , (currently amended) A tbree-dtmeiisioiial television, system, comprising: 



2 an acquisition stage, comprising: 

3 a plurality of video cameras, each video camera configured to acquire 

4 a video of a dynamically changing scene in real-time; 

5 means for synchronizing die plurality of video cameras; and 

6 a plural ity of producer modules connected to the plurality of video 

7 cameras, the producer modules configured to compress the 

8 videos to compressed videos and to determine viewing 

9 parameters of the plurality of video cameras , in which the 

10 viewing parameters include a position, orientation, field-of- 

1 1 view, and focal plane of each video camera : 

12 a display stage, comprising: 

.13 a pl urality of decoder modules con figured to decompress the 

14 compressed videos to uncompressed videos; 

1 5 a pl urality of consumer modules con figured to generate a plural ity of 

1 6 output videos from the decompressed videos according to the 

17 viewing parameters ; 

18 a controller configured to broadcast the viewing parameters to the 

1 9 plurality of decoder modules and the plurality of consumer 

20 modules; 

21 a three-dimensional display unit configured to concurrently display 

22 the plurality of output videos onto a single display surface 

23 acco r d ing to the viewing p a rameters ; and 
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24 means for connecting the plurality of decoder modules, the plura I ity 

25 of consumer modules, and the three-dimensional display unit; 

26 and 

27 a transmission stage, connecting the acquisition stage to the display stage, 

28 configured to transport the plurality of compressed videos and the viewing 

29 parameters. 

1 2. (currently amended ) The system, of claim 1, further comprising a plurality of 

2 cameras eon figured to acquire calibration images displayed on the display surface 

3 of the three-dimensional display unit to determine the viewing parameters. 

I 3 . (original) The system of claim 1 , in which the display units are projectors. 

1 4. (original) The system of claim 1 , in which the display units are organic light 

2 emitting diodes. 

1 5, (original) The system of claim 1, in which the three-dimensional display unit 

2 uses front-projection. 

.1. 6. (original) The system of claim .1, in which the three-dimensional display unit 

2 uses rear-projection. 

1 7. (original) The system of claim 1, in which the display unit uses two-dimensional. 

2 display element. 
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1 8. (previously presented) The system of claim 1, in which the display unit uses a 

2 flexible fabric, 

9. (canceled) 

1 10, (original} The system of claim 1, in which different output images are 

2 displayed depending on a viewi ng direction of a viewer. 

1 11. (previously -presented) The system of claim 1 , in which static view-dependent 

2 images of an environment are displayed such that a display surface of the display 

3 unit: disappears. 

1 12. (previously presented) The system of claim 1, i n which dynamic view- 

2 dependent images of an environment: are displayed such that a display surface of 

3 the display unit disappears. 

1 13. (original) The system of claim 11 or 12, in which the view-dependent images 

2 of the environment are acquired by a plurality of cameras. 

1 1 4. (original) T he system of claim 1, in which each producer module is connected 

2 to a subset of the plurality of video cameras. 

1 1 5, (original) The system of claim I. in which the plurality of video cameras are in 

2 a regularly spaced linear and horizontal array. 
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1 16, (original) The system of claim l s in which the plurality of video cameras are 

2 arranged arbitrarily. 

1 17. (original) The sy stem of claim 1 , in which an optical axis of each video camera 

2 is perpendi c u i ar to a common plane, and the op vectors of the plurality of video 

3 cameras are vert ically aligned. 

1 18. (original) The system of claim 1, in which the view ing parameters incl ude 

2 intrinsic and extrinsic parameters of the video cameras. 

1 19. (original) The system of claim I , further comprising: 

2 means for selecting a subset of the plurality of cameras for acquiring a subset 

3 of videos. 

1 20. (original) The system of claim L in which each video is compressed 

2 individually and temporally. 

2.1. (canceled) 

1 22, (previously presented) The system of claim 1, in which the controller 

2 determines, for each output pixel o(u, v) in the output video, a view number v and a 

3 position of each source pixel s(v t x, y) in the decompressed videos that cont ributes 

4 to the output pixel in the output video. 
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1 23, (original) The system of claim 22, in which the output pixel is a linear 

2 combination of k source pixels according to 

3 o(u , v) = ]T w/( v > a*, y ) , 

Ml 

4 where blending weights w f are predetermined by the controller based, on the 

5 viewing parameters. 

1 24. (original) The system of claim 22, in which a block of the source pixels 

2 contribute to each output pixel. 

1 25. (original) The system of claim I , In which the three-dimensional display unit 

2 includes a display-side lenticular sheet, a viewer-side lenticular sheet:, a diffuser, 

3 and substrate between each lenticular sheets and the diffuser. 

1 26. (original) The system of claim 1 , in which the three-dimensional display unit 

2 includes a display-side lenticular sheet, a reflector, and a substrate between the 

3 lenticular sheets and the reflector. 

1 27. (previously presented) The system of claim I, in which an arrangement of the 

2 cameras and an arrangemen t, of t he display units, with respect to the di splay unit, 

3 are substantially identical, and the number of cameras and the number of display 

4 units is greater than two. 

1 28. (previously presented) The system of claim 1, in which the plurality of cameras 

2 acquire the video of high dynamic light-fields. 
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29. (previously presented) The system .of claim 1, in which the display units 
display the output videos as high dynamic light-fields. 
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30. (canceled) 



31. (canceled) 
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Remarks 

Claims 1-8 and J 0-29 are pending in the application. Claims 1-8 and 10-29 are 
rejected. All rejections are respectfully traversed 

Claims 1-3, 5-24 and 27 are rejected under 35 U.S.C, 1 02(b) as being anticipated 
by Ritchey, U.S. Patent No. 5,495,576 (Ritchey). 

Ritchey describes virtual model-based virtual image rendering. Visual information 
is acquired using a camera, and shape information is acquired using radar (column 
10, lines 1.7-25). Following acquisition of the sensor signals, Ritchey constructs a 
3D virtual model (see Abstract, " The computer processor integrates the sensor 
signals, processes signals as a virtual model. . .** and column 9, lines 6-14). 
Depending on the actions of a user, the 3D model is updated, for example, by 
moving a virtual object or changing the viewpoint of the display (column 9, lines 
46-58), and virtual images are generated from the virtual model and rendered on a 
display (column 20, lines 46-50). 

Ritchey generates a virtual model from sensor signals, and then generates virtual 
images from the virtual model. The claimed output video is generated from a 
compressed version of a video acquired of a dynamically changing scene. 

Regarding claim 1 , Ritchey fails to teach the claimed producer modules configured 
to determine viewi ng parameters of the plurality of video cameras, in which the 
viewing parameters include a position, orientation, fleld-of-view, and focal plane, 
of each video camera. 
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As the displayed virtual images of Ritchey are entirely virtual, model-based, there 
is no need for Ri tchey to determine viewing parameters of video cameras, in which 
the viewing parameters include a position, orientation, field-of-view, and focal 
plane, of each video camera. All of the information Ritchey requires to display an 
image Is determined by sampling the 3D model (column 2, lines 30-39). There is 
no mention of determining a position, orientation, field-of-view, or focal plane of a 
camera anywhere in Ritchey. At column 1 \ > lines 55-61, and column 1.2, lines 1-9, 
Ritchey describes internal (intrinsic) camera parameters, such as chrominance, 
luminance, hue, and intensity, and additional hardware components that may be 
added to a camera. These are not viewing parameters, and certainly are not a 
position, orientation, field-of-view, and focal plane of each camera as claimed. 

in addition, Ritchey fa ils to teach the claimed plurality of consumer modules 
configured to generate a plurality of output videos from the decompressed videos 
according to the viewing parameters. 

As explained above, Ritchey fails to teach claimed producer modules configured to 
determine viewing parameters of the plurality of video cameras, in which the 
viewing parameters include a position, orientation, field-of-view, and focal plane, 
of each video camera. T herefore, Ritchey cannot teach a plurality of consumer 
modules configured to generate a pl urality of output videos from the decompressed 
videos according to the viewing parameters. 

Furthermore, all of the information Ritchey requires to di splay his virtual image is 
determined by sampling the 3D virtual model. Depending on the act ions of a user. 
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Ritchey updates and samples the 3D virtual model to determine what virtual 
images to display (column 20, lines 46-56). At column 22, lines 45-49, Ritchey 
describes determining a perspective projection relative to a user's head and eyes. 
This is not equivalent to generating a plurality of output videos according to the 
viewing parameters. As claimed, viewing parameters include position, orientation, 
field-of-view, and focal plane of each video camera used to acquire the videos, and 
not of a user viewing the videos. Ritchey cannot anticipate the claimed in vention 

Regarding claim 2 . Ritchey fails to teach a plurality of cameras configured to 
acquire calibration images displayed on the display surface of the three- 
dimensional display unit to determine the viewing parameters. At column 1.1, lines 
61-63 , Ritchey describes compressing a plurality of images into a single frame. 
Examples of these single frames are shown in Figs. 9a, 9b, and 10 of Ritchey, see 
also column 12, lines 30-34 and lines 49-54. These images cannot be used to 
determine viewing parameters including position, orientation, field-of-view, and 
focal plane of each video camera. Additional ly, these images are not calibration 
images. Calibration images, as known in the art, are used to compare a device or 
the output of an instrument to a standard having known measurement 
■characteristics. In an embodiment of the invention, calibration images are used to 
determine viewing parameters for generating a plurality of output videos, see 
Specification, paragraphs [G92]-[094]. There is nothing in Ritchey that teaches the 
use of calibration images as claimed. 

Regarding claims 3 and 5-8, Ritchey fails to teach the claimed producer modules 
configured to determine viewing parameters of the plurality of video cameras, in 
which the viewing parameters include a position, orientation, field-of-view, and 
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focal plane, of each vi deo camera, and the claimed plurality of consumer modules 
configured to generate a plurality of output videos from the decompressed videos 
according to the viewing parameters. Therefore, Ritchey cannot anticipate the 
claimed invention further comprising the display units as claimed in claims 3 and 
5-8. 

Regarding claims 10-13, Ritchey fails to teach the claimed producer modules 
configured to determine viewing parameters of the plurality of video cameras, in 
which the viewing parameters include a position, orientation, fieid-of-view, and 
focal plane, of each video camera, and the claimed plura lity of consumer modules 
configured to generate a plurality of output videos from the decompressed videos 
according to the viewing parameters. Therefore, Ritchey cannot anticipate the 
claimed invent ion further comprising the display of view-dependent images as 
claimed in claims 10-13. 

Regarding claim 14, Ritchey fails to teach the claimed producer modules 
configured to determine viewing parameters of the plurality of video cameras, in 
which the viewing parameters include a position, orientation, field-of-view, and 
focal plane, of each video camera, and the claimed pl urality of consumer modules 
configured to generate a plurality of output videos from the decompressed v ideos 
according to the viewing parameters. Therefore, Ritchey cannot anticipate the 
claimed invention further comprising producer modules connected to a subset of 
the plurality of video cameras as claimed in claim 14. 

Regarding claims 15-17, Ritchey fails to teach the clai med producer modules 
configured to determine vie wing parameters of the plurality of video cameras, in 
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which the viewing parameters include a position, orientation, field-of-view, and 
focal plane, of each video camera, and the claimed plurality of consumer modules 
configured to generate a plurality of output videos from the decompressed videos 
according to the viewing parameters. Therefore, Ritchey cannot anticipate the 
claimed invention further comprising the arrangement of video cameras as claimed 
in claims 15-17. 

Furthermore, the cameras depicted in Fig. .1 7 are not in a regularly spaced linear 
and horizontal array as claimed in claim 15. Each camera in that figure is shown to 
be acquiring a different side of an object, see "Side A" through "Side F" clearly 
marked in. Figure 1 7; column 10, lines 32-34; and Fig. 6. This is impossible with a 
regularly spaced linear and horizontal array as claimed in claim 15, 

At column. 1 2, lines 44-47, Ritchey describes optical fiber bundles used to transmit 
images "of various sides of the subject (see lines 32-34) to the light sensitive 
recording surface of a camera . Acquiring images of various sides of an object, as 
described in Ritchey, requires a camera arrangement that can receive light from all 
sides of an object. This cannot be done with the camera arrangement as claimed in 
claim 17. Although the fiber optic cables can be twisted to achieve the described 
alignment/or the cables themselves, this does not teach an optical axis of each 
video camera perpendicular to a common plane, and the up vectors of the plurality 
of video cameras being vertically aligned as claimed in claim 17. Again, it is clear 
that Ritchey does not describe a linear or quasi-linear array of cameras as claimed. 

Regarding claim 1 8, Ritchey tails to teach the claimed producer modules 
configured to determine viewing parameters of the plurality of video cameras, in 
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which the viewing parameters include a position, orientation, field-of-view, and 
focal plane, of each video camera, and the claimed plurality of consumer modules 
configured to generate a plurali ty of output videos from the decompressed videos 
according to the viewing parameters. Therefore, Ritchey cannot anticipate the 
claimed invention further comprising the viewing parameters as claimed in claim 
1 8, Furthermore, as explained above, Ritchey does not teach extrinsic parameters, 
i.e., parameters determined by camera placement as opposed to internal parameters 
determined by camera hardware, at column 1 1 , lines 55-63, and column 12, lines 
3-11. 

Regarding claim 1 9, Ritchey tails to teach the claimed producer modules 
configured to determine viewing parameters of the plurality of video cameras, in. 
which the viewing parameters include a position, orientation, field-of-view, and 
focal plane, of each video camera, and the claimed plurality of consumer modules 
configured to generate a plurality of output videos from the decompressed videos 
according to the viewing parameters. Therefore, Ritchey cannot anticipate the 
claimed invention further comprising the selection of a subset of cameras as 
claimed in claim 19. 

Regarding claim 20 s Ritchey fails to teach the claimed producer modules 
configured to determine viewing parameters of the plurality of video cameras, in 
which the viewing parameters include a position, orientation, field-of-view, and 
focal plane, of each video camera, and the claimed plurality of consumer modules 
configured to generate a plurality of output videos from the decompressed videos 
according to the viewing parameters. Therefore, Ritchey cannot anticipate the 
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claimed invention further comprising the compression of videos as claimed in 
claim 20. 

Regarding claims 22-24, Ritehey fails to teach the claimed producer modules 
configured to determine viewing parameters of the plurality of video cameras, in 
which the viewing parameters include a position, orientation, field-of-view, and 
focal plane, of each video camera, and the claimed plurality of consumer modules 
configured to generate a plurality of output videos from the decompressed videos 
according to the viewing parameters. Therefore, Ritehey cannot anticipate the 
claimed invention further comprising the output pixel processing as claimed in 
claims 22-24. 

Regarding claim 27, Ritehey fails to teach the claimed producer modules 
configured to determine viewing parameters of the plurality of video cameras, in 
which the viewing parameters include a position, orientation, field-of-view, and 
focal plane, of each video camera, and the claimed plurality of consumer modules 
configured to generate a plural i ty of output videos from t he decompressed videos 
according to the viewing parameters. Therefore, Ritehey cannot anticipate the 
claimed invention further comprising the arrangement of video cameras and 
display unit as claimed in claim 27. 

Claim 4 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over Ritehey. 

Regarding claim 4, Ritehey fails to teach the claimed producer modules configured 
to determine viewing parameters of the plurality of video cameras, in which the 
viewing parameters include a. position, orientation, field-of-view, and focal plane. 
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of each video camera, and the claimed .plurality of consumer modules configured 
to generate a plurality of output videos from the decompressed videos according to 
the viewing parameters. Therefore, Ritchey cannot anticipate the claimed invention 
further comprising organic light emitting diodes as claimed in claim 4. 

Chums 25 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ritchey, in view of Ezra et aL U.S. Patent No. 5,703,7 1 7 (Ezra). 

Regarding claims 25 and 26, me combination of Ritchey and Ezra fails to leach the 
claimed producer modules configured to determine viewing parameters of the 
plurality of video cameras, m which the viewing parameters include a position, 
orientation, fleld-of-view, and focal plane, of each video camera, and the claimed 
plurality of consumer modules configured to generate a plurality of output videos 
from the decompressed videos according to the viewing parameters. Therefore, 
Ritchey cannot anticipate the claimed invention farther comprising the lenticular 
display units as claimed in claims 25 and 26. 

Claims 28 and 29 are rejected under 35 ILS.C. 103(a) as being unpatentable over 
Ritchey, in vie w of Nayar et a!., U.S. Patent Application Publicat ion No. 
2004/0070565 (Nayar). 

Regarding claims 28 and 29, the combination of Ritchey and Nayar fails to teach 
the claimed producer modules configured to determine viewing parameters of the 
plurali ty of video cameras, in which the viewing parameters incl ude a position, 
orientation, field-of-view, and focal plane, of each video camera, and the claimed 
plurali ty of consumer modules configured to generate a plurality of output videos 
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from the decompressed videos according to the viewing parameters. Therefore, 
Ritchey cannot anticipate the claimed invention further comprising the acquisition 
and display of high dynamic lightfields as claimed in claims 28 and 29. 



It is believed that this application is now in condition for allowance. A notice to 
this effect is respectfully requested. Should further questions arise concerning this 
application, the Examiner is invited to call Applicant's attorney at the number 
listed below. Please charge any shortage in fees due in connection with the filing of 
this paper to Deposit Account 50-0749. 

Respectfully submitted, 

Chiton D. Mueller/ 



Clifton D. Mueller 
Attorney for the Assignee 
Reg. No. 57,836 

Mitsubishi Electric Research Laboratories, Inc. 
20.1 Broadway, 8 tli Floor 
Cambridge, MA 02 139 
Telephone: (617) 621-7517 
Customer No. 22199 
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